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Abstract 

Introduction and Objective. This paper provides an overview of the DUQuE (Deepening our Understanding of Quality 
Improvement in Europe) project, the first study across multiple countries of the European Union (EU) to assess relationships 
between quality management and patient outcomes at EU level. The paper describes the conceptual framework and methods 
applied, highlighting the novel features of this study. 

Design. DUQuE was designed as a multi-level cross-sectional study with data collection at hospital, pathway, professional and 
patient level in eight countries. 

Setting and Participants. We aimed to collect data for the assessment of hospital-wide constructs from up to 30 randomly 
selected hospitals in each country, and additional data at pathway and patient level in 12 of these 30. 

Main outcome measures. A comprehensive conceptual framework was developed to account for the multiple levels that influ- 
ence hospital performance and patient outcomes. We assessed hospital-specific constructs (organizational culture and profes- 
sional involvement), clinical pathway constructs (the organization of care processes for acute myocardial infarction, stroke, hip 
fracture and deliveries), patient-specific processes and outcomes (clinical effectiveness, patient safety and patient experience) and 
external constructs that could modify hospital quality (external assessment and perceived external pressure). 

Results. Data was gathered from 188 hospitals in 7 participating countries. The overall participation and response rate were 
between 75% and 100% for the assessed measures. 

Conclusions. This is the first study assessing relation between quality management and patient outcomes at EU level. The study 
involved a large number of respondents and achieved high response rates. This work will serve to develop guidance in how to 
assess quality management and makes recommendations on the best ways to improve quality in healthcare for hospital stake- 
holders, payers, researchers, and policy makers throughout the EU. 



^The list of authors of DUQuE Project Consortium is given in the Appendix. 
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Introduction 

Research on quality in health care has built a sizeable knowl- 
edge base over the past three decades. Evidence on the effect- 
iveness of organizational quality management and improvement 
systems has emerged more recently [1—4]. This recent research 
has addressed several questions. Do quality improvement activities 
lead to better quality of care? Which quality improvement tools 
are most effective? How can various quality tools be integrated 
into a sensitive and effective quality and safety improvement 
programme? What factors impact on the implementation of 
quality strategies at hospital level? [5, 6]. 

Recent European Union (EU) projects, such as the 'Methods 
of Assessing Responses to Quality Improvement Strategies' 
(MARQUIS) and the 'European Research Network on Quality 
Management in Health Care' (ENQUAL), have attempted to 
measure the effects of a variety of quality strategies in European 
hospitals in order to provide information that payers could 
use as they contract for the care of patients moving across 
borders. This prior research has had limitations including an 
incomplete conceptualization of antecedents and effects of 
quality management systems, methodological weaknesses in 
measurement strategies and limited use of data on patient 
outcomes in the evaluation of the effectiveness of these strat- 
egies [7-12]. 

This paper provides an overview of the conceptual frame- 
work and methodology of the recently completed DUQuE 
project (Deepening our Understanding of Quality Improvement 
in Europe) funded by the European Commission 7th 
Framework Programme. At its start, the goal of this project 
was to evaluate the extent to which organizational quality im- 
provement systems, organizational culture, professional in- 
volvement and patient involvement in quality management are 
related to the quality of hospital care, assessed in terms of clin- 
ical effectiveness, patient safety and patient experience in a 
large and diverse sample of European hospitals. 

DUQuE had the following specific objectives: 

(1) To develop and validate an index to assess the imple- 
mentation of quality management systems across 
European hospitals (addressed in three papers by 
Wagner et al. [13-15]). 

(2) To investigate associations between the maturity of quality 
management systems and measures of organizational 
culture, professional engagement and patient involvement 
in quality management, including board involvement on 
quality and its association with quality management 
systems (addressed by Botje et al. [16]), the relationship 
between organizational culture and structure and quality 
management systems (addressed by Wagner et al. [17]) 
and the development and validation of the measures for 
evaluation of clinical management by physicians and 
nurses (addressed by Plochg et al. [18]). 



(3) To investigate associations between the maturity of 
quality management systems and patient level measures 
(PLMs) of clinical effectiveness, patient safety and 
patient experience including the relationships between 
quality management systems at hospital and pathway 
levels (addressed by Wagner et al. [19]), patient safety 
and evidence based management (addressed by Sunol 
et al. [20]), and patient involvement in quality management 
(addressed by Groene et al. [21]). 

(4) To identify factors influencing the uptake of quality 
management activities by hospitals, including external 
pressure as enforced by accreditation, certification or 
external assessment programmes (addressed by Shaw 
et al. [22]) and the feasibility of using routine data to 
compare hospital performance (addressed by Groene 
etal. [23]). 

Conceptual framework 

The DUQuE study conceptual model was developed by a 
range of collaborators with a diverse range of disciplinary 
backgrounds and expertise. The model addresses four levels: 
hospital, departmental (or pathway) level, patient's level and 
external factors influencing uptake of management decisions 
(Fig. 1). We hypothesized that hospital level quality manage- 
ment systems would be associated with hospital governance 
and culture, the degree of professional involvement in man- 
agement (and more concretely in quality management) and the 
degree of patient involvement in quality management. We 
further hypothesized that quality management systems in place 
to manage specific clinical pathways (sometimes referred to as 
'clinical service lines') would be associated with quality man- 
agement activities at the hospital and pathway level as well as 
with pathway culture, professionalism and patients' involve- 
ment in quality management. Moreover, we expected quality 
management systems at both hospital and clinical pathway 
levels to be associated with patients' experience, and the safety 
and clinical effectiveness of patient care for four selected con- 
ditions (acute myocardial infarction (AMI), obstetrical deliver- 
ies, hip fracture and stroke). These conditions were selected 
because they cover an important range of types of care, there 
are evidence-based standards for process of care against which 
compliance could be assessed and there is demonstrated vari- 
ability in both compliance with process of care measures and 
outcomes of care (complications, mortality) that would allow for 
analysis of associations between these measured constructs. 
Finally, the conceptual model posited that factors external to 
these quality management systems would also influence the 
uptake of quality management activities such as perceived pres- 
sure from hospital leadership (chief executive officers and gov- 
ernance boards) and external accreditation, certification and 
standards programmes that offer audit and feedback regarding 
quality management and performance. 
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Figure I DUQuE conceptual framework. 

Development of measures 

For each construct in the conceptual model, published litera- 
ture was searched to identify measure sets and instruments 
used previously. Explicit criteria were used to assess and 
select measures, including psychometric properties, level of 
evidence and appropriateness in multi-national studies. 
Whenever a validated measure, instrument or measure set 
existed and was deemed appropriate, we requested permis- 
sion to use it in the DUQuE study. Where we found no exist- 
ing measures or instruments, new measures were developed. 
After synthesis, 25 measure domains were defined and 
included in DUQuE (Table 1). Measures were originally 
designed in English language and translated to local lan- 
guages when necessary. Table 2 shows the number of cases 
expected and finally gathered for each instrument designed 
to operationalize measures according to hospital type. All of 
the final instruments used are available at the DUQuE's 
website [24]. 

The DUQuE project used a variety of measures to assess 
patient experience including a generic patient experience in- 
strument (NORPEQ), a measure of patient-perceived dis- 
charge preparation (Health Care Transition Measure) and a 
single-item measure for perceived involvement of care and hos- 
pital recommendation in a DUQuE patient survey instrument. 
In addition, clinical indicators to assess key patient processes 
and outcomes were selected using a multi-step process (Fig. 2) 
that included a search of peer-reviewed literature and extensive 
searches of guidelines and other sources (a substantial amount 
of patient safety literature appears in the so-called 'grey litera- 
ture' and not in peer-reviewed publications). 



Clinical indicators for each of the four clinical conditions 
included in the study were collected, reviewed and rated based 
on the level of evidence according to the Oxford Centre of 
Evidence-Based Medicine [25] and their previous use in multi- 
national studies. In the next step, five relevant European scien- 
tific societies, five individual key experts with knowledge of the 
health-care systems in Europe and the eight DUQuE field test 
coordinators, named country coordinators, took part in a multi- 
stakeholder panel. Participants assessed the proposed clinical 
indicators using a rating matrix. Criteria applied to select four to 
six clinical indicators of process and outcome per condition 
were: comparability, data availability, data quality across the eight 
countries participating, relevance in the different EU health-care 
settings and ability of the PLM to distinguish between hospitals. 
The country coordinators offered structural and epidemiologic- 
al information about treatment of each of the four clinical con- 
ditions. The final indicators were selected from among those 
with the highest level of evidence and the best multi- stakeholder 
panel rating results with previous use of an indicator in clinical 
cross-national comparisons given special consideration. 

In each condition one or two composite measures were devel- 
oped that combined the selected indicators for the purpose of a 
global analysis of the condition. Table 3 shows the final DUQuE 
indicator list with the source and level of evidence rating for 
each. Data from clinical indicators were collected via medical 
records abstraction using a standardized data collection method. 

Design and fieldwork protocol 

DUQuE was designed as a multi-level cross-sectional study 
with data collection in eight countries. Selected countries had 



7 



Secanell et al. 



Table I Constructs, measure domains and data collection methods used 
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Table I Continued 
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Table I Continued 



Construct name 



Measure domain 



Measure domain definition Data collection method Administration system 
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Hospital 

recommendation 

Perceived continuity 
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(Health Care Transition 
Measure) 

Measures patients' 
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to have a sufficient number of hospitals to fulfil sampling cri- 
teria, represent varied approaches to financing and organizing 
health care, have research staff with experience in conducting 
comprehensive field tests and represent the geographical reach 
of the EU. Turkey was included because of the status of its EU 
candidacy at the start of the project. The countries invited to 
participate in the field test were the Czech Republic, England, 
France, Germany, Poland, Portugal, Spain and Turkey. 

Sampling strategy 

We aimed to collect data for the assessment of hospital-wide 
constructs from up to 30 randomly selected hospitals in each 
of the 8 participating countries (NJ, = 240 hospitals, maximum). 
In each country, for 12 of these 30 hospitals we carried out com- 
prehensive data collection at pathway and patient levels = 96 
hospitals, maximum). 

General acute care hospitals (public or private, teaching or 
non- teaching) with a minimum hospital size of 130 beds were 
considered for inclusion into the study. In addition, for compre- 
hensive data collection, hospitals were required to have a suffi- 
cient volume of care to ensure recruitment of 30 patients per 
condition over a 4-month period (a sample frame of a minimum 
of 90 patients). Specialty hospitals, hospital units within larger 
hospitals, and hospitals not providing care for the four clinical 
conditions of study were excluded from further consideration. 

Each country coordinator provided to the Central Project 
Coordination Team (CPCT) a de-identified hospital list (includ- 
ing all hospitals with more than 130 beds, ownership and teach- 
ing status) of each country for sampling purposes. A simple 
random sample was taken to identify candidate hospitals for 
comprehensive and partial data collection; this was oversampled 
to anticipate withdrawal of participants. The sampling strategy 



proved efficient as the distribution of key characteristics such as 
number of beds, ownership and teaching status did not differ 
between country-level samples and national distributions. 

Recruitment 

After the preliminary sample of hospitals was selected, each 
country coordinator formally invited the hospital chief executive 
officers (CEOs) to participate. A total of 548 hospitals were 
approached of which 192 agreed to participate. The percentage 
of hospitals accepting participation ranged from 6.7 to 96.8% 
between countries. The main reasons of declining to participate 
were related to time constraints, organizational aspects and the 
complexity of the study. Data from 188 hospitals in 7 participat- 
ing countries were included in the final analysis. After significant 
efforts, hospitals in England were not included partly due to 
delays in obtaining ethical approval and also extensive difficulty 
recruiting hospitals. 

Data collection 

The CPCT facilitated communication and problem solving 
during the ongoing field test process. Preparation began in July 
2010 and included sampling, recruitment, back and forward 
translation and piloting of measures, preparation of materials 
and training hospital coordinators in charge of retrieving data. 
Data collection began formally in May 201 1 and was completed 
by February 2012. Several coordination actions enabled timely 
data collection of professional questionnaires, patients' surveys, 
charts review, external visits and administrative data (Fig. 3). 

An information technology platform was created to support 
central collection of responses to professional questionnaires 
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Table 2 Data collection and response rates in the DUQuE project 



Type of 
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Expected n 
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questionnaire 


questionnaires per 
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partial data 
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questionnaires per 
hospital with 
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External visits 


0 


1 
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74 
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Administrative 


1 


1 


188 


177 


94 


77-100 


routine data 














Total amount 


26 


371 


29 806 


25 940 


87% 





"By country, % response rate range (minimum— maximum). 



and data from site visits. Among a subset of 88 hospitals a com- 
prehensive data collection required them to provide administra- 
tive data and administer 371 questionnaires including patients, 
professionals, chart reviews and an external visit (Table 2). The 
remaining hospitals provided administrative data and adminis- 
tered professional questionnaires to 25 respondents. The overall 
response rate for the different questionnaires was between 75 
and 100% for the assessed measures (Table 2). 

Once field period was completed, all data collected was cross- 
checked by the CPCT to ensure that hospital and respondents' 
codes matched per country. Data were also checked for any dis- 
crepancies, specifically the chart review files. All data sets were 
merged and transferred to the data analysis team. 

Ethical and confidentiality 

DUQuE fulfilled the requirements for research projects as 
described in the 7th framework of EU DG Research [26]. 
Data collection in each country complied with confidentiality 



requirements according to national legislation or standards of 
practice of that country. 

Discussion 

The DUQuE project has achieved several important advances 
that are detailed in the papers included in this supplement. 
First, it is the first large-scale quantitative study exploring 
across a broad and diverse sample of European hospitals 
several features of the implementation of hospital quality man- 
agement systems and their impact on quality and safety. It 
represents a major advance over previous research by includ- 
ing patient-level processes and outcomes, enabling study of 
the impact of current quality improvement strategies on 
patient care and outcomes. Secondly, it made use of a compre- 
hensive conceptual model describing the functioning of hos- 
pital quality management systems and supporting a deliberate 
empirical, quantitative measurement and analytic approach. 
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Figure 2 Development process of DUQuE clinical indicators. 



Thirdly, it succeeded in collecting and linking large amounts of 
data from hospitals in seven countries including professional 
surveys (management and baseline), patient surveys, clinical 
indicators from medical records and administrative data from 
participating hospitals. Fourthly, the DUQuE coordination 
structure and protocol including periodic follow-up and feed- 
back to hospital coordinators enabled the project to obtain the 
active participation of hospitals and high response rates. 
Finally, the project led to the development of new measures, 
reported in this supplement, which can be used in future 
research. 

Despite these many achievements, the study has limitations 
that are worth highlighting. First, the study was not designed 
or powered to report on the results for each country. We 
pooled data across countries and addressed heterogeneity in 
country-level estimates in our statistical modelling using an ap- 
proach that allowed country-level baselines and effects to vary. 
Further covariate adjustment included hospital size, ownership 
and teaching status. A second limitation is that we used a 
cross-sectional study design. This was considered most appro- 
priate for hypothesis testing in this project because it requires a 
relatively shorter time commitment and fewer resources to 
conduct, although ultimately does not allow strong conclu- 
sions about causality [27]. The use of directed-acyclic graphs 
guided the development of our statistical models, incorporat- 
ing theory and knowledge derived from previous research 
findings and this approach helps somewhat in strengthening 
causal inferences from the project results. A third limitation is 
related to the sampling strategy. Although a random sampling 
strategy was used, a generalization to participating countries 
and hospitals is limited because hospitals volunteered to par- 
ticipate in the project introducing potential self-selection bias. 
However, our analysis shows substantial variations in the im- 
plementation of quality criteria, and outcomes, suggesting that 
the effect of self-selection may be less problematic. A final 
limitation is the existence of systematic biases due to self- 
report and to country differences in data coding. The study 



relied on self-reported data that may introduce social desirabil- 
ity bias. In many instances, we were able to use other data 
sources to verify responses, but not for all measures. 
Measuring clinical practice using clinical record review may 
introduce systemic biases related to medical record complete- 
ness practice variations among countries, but some of this is 
handled through statistical adjustment for the effects of 
country, hospital size, ownership and teaching status. In con- 
clusion, while the design is prone to a number of limitations 
we anticipated potential problems and addressed them in 
study design and our analytical strategy. 

This supplement presents the first set of results from the 
DUQuE project. It includes quality management indexes con- 
ceptualization and development, validation of new measures 
and results of tests of associations between some of the main 
constructs. Future publications are intended to address re- 
maining questions and secondary analyses of this very large 
database. For example, papers addressing the associations 
between quality management systems and key patient pro- 
cesses and outcomes are planned. In addition to the research 
publications, the DUQuE project website offers a set of 
recommendations presented in an appraisal scheme derived 
from the broader perspective of this investigation and sum- 
marized for hospital managers buyers and policy makers who 
may find it useful as they redesign organization-wide 
approaches to hospital care to achieve the highest levels of 
quality and safety [24] . 
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Table 3 Clinical indicators and composite measures selected for the DUQuE project 



Condition 



Main clinical indicators used 



Source 2 



Level of 
evidence 



Acute myocardial 
infarction (AMI) 



Deliveries 



Hip fracture 



Stroke 



Fibrinolytic agent administered within 75 min of hospital arrival 
Primary percutaneous coronary intervention within 90 min 
Thrombolytic therapy or primary percutaneous coronary 
intervention given 
Therapy given on time b 

Anti-platelet drug prescribed at discharge (ASPRIN) 

Beta blocker prescribed at discharge 

Medication with statin prescribed at discharge 

ACE inhibitors prescribed at discharge 
Appropriate medications: all four of anti-platelet, beta blocker, 
statin, ACE inhibitor) prescribed at discharge b 
Epidural anaesthesia applied within 1 h after being ordered for 
vaginal births 

Exclusive breastfeeding at discharge 

Blood transfusion during intended or realized vaginal birth 

Acute Caesarean section 

Obstetric trauma (with instrumentation) 

Obstetric trauma (without instrumentation) 

Mother complication: unplanned C-section, blood transfusion, 

laceration and instrumentation 13 

Adverse birth outcome (child)' 3 

Birth with complications 13 

Prophylactic antibiotic treatment given within 1 h prior to surgical 
incision 

Prophylactic thromboembolic treatment received on the same day 
as admission (within 24 h or on same date when (one or more) 
times not provided) 

Early mobilization (within 24 h or before next day when (one or 
more) times not provided) 

In-hospital surgical waiting time <48 h [or 1 day when (one or 
more] times not provided) 

Percentage of recommended care per case: indicators prophylactic 
antibiotic treatment within 1 h, prophylactic thromboembolic 
treatment within 24 h, early mobilization within 24 h, in-hospital 
surgical waiting time <48 h = YES) b 

Admitted to a specialized stroke unit within 1 day after admission 

Platelet inhibitor treatment within 2 days after admission 

Diagnostic examination within first 24 h/ same day after admission 
using CT or MRI scan 

Mobilized within 48 h (or 2 days when times are missing) after 
admission? 

Appropriate stroke management. All three applied: Platelet 
inhibitor treatment within 2 days after admission, Diagnostic 
examination (CT or MR!) within first 24 h and Mobilized within 
48 h b 



AHRQ 
AHRQ 

See la and 2a 



A 

A/B 

See la and 
2 a 



AHRQ 


A 


AHRQ 


A 


AHRQ 


A 


AHRQ 


A 


The Danish 


D 


Clinical Registries 




WHO 


D 


The Danish 


B 


Clinical Registries 




OECD 


B 


OECD 


B 


The Danish 


B 


Clinical Registries 




The Danish 


B 


Clinical Registries 




The Danish 


B 


Clinical Registries 




RAND 


A 


RAND 


A 


The Danish 


B 


Clinical Registries 




OECD 


C 



The Danish A 
Clinical Registries 
The Danish A 
Clinical Registries 
The Danish D 
Clinical Registries 
The Danish C/D 
Clinical Registries 



"According the Oxford Centre of Evidence-Based Medicine. 
''Composite measures (aggregation of indicators). 
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1. Hospital sampling and 

recruitment 
(July-Desember 2010) 



2. Measures piloting and 

translation 
(July 2010- February 2011) 



3. Training and preparation 
for data collection 
(January-April 2011) 



4. Data collection (May 2011-February 2012) 



Comprehensive data collection at hospital and pathway level (n=371) 



Partial data collection (n=26) 



At Hospital level: 

■ Questionnaire to hospital 
managers (n=5) 

■ Questionnaire to leading 
professionals at hospital level 
(n=20) 

■ Administrative data (n=l) 



At Pathway level for Acute Myocardial infarction, 
Stroke, Hip Fracture and Deliveries: 

■ Questionnaires to managers of the pathway 
(n=4) 

■ Questionnaires to professionals of the pathway 
(n=80) 

■ Patient Survey (n=120) 

■ Charts review (n=140) 

■ External visit to hospital and pathways (n=l) 



5. Data cleaning, merging for analysis (May-October 2012) 



Figure 3 Stages of DUQuE field test. 
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the HAS team: L. Banaei-Bouchareb, Pilot Programme- 
Clinical Impact department, HAS, N. Danchin, SFC, French 
cardiology scientific society, Past President, J.M. Davy, SFC, 
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